Transformer Engine

NVIDIA released BioNeMo Recipes, a set of training recipes that use Low-Rank Adaptation (LoRA) to fine-tune large biological foundation models like ESM2-3B and Evo2-1B on a single workstation GPU. The…

NVIDIA introduced advanced fused MLP kernels for mixture-of-experts (MoE) models, built with the CuTe DSL, delivering 1.3x–2x kernel-level speedups and enabling sync-free MoE execution. The optimizati…